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ABSTRACT 

An implementation technique for functional languages that has fcceived recent attention is grapk 
reduction, which offers opportunity for the eipkatation ai parallelim by multiple processors. 
While several proposals for parallel graph reduction machines have been made, differing terminol- 
ogy and approaches make these proposals difficult to compare. This paper presents a systematic 
approach to the study of parallel graph reductimi machines, and proposes an abstract architecture 
for such a machine that is independent of the base language and ccnnmunication network chosen 
for an actual implementation. The abstract architecture, in additkn to serving as a foundation tor 
the design of real machines, lends quite a bit al insight into the essence of parallel graph reduc- 
tion. 



Keywords: Abstract Machines, Applicative Languages, Ccnoputer Architecture, Multiple Pro- 
cessor Architectures, Parallel Procesung, Redocticm. 
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1. Intredactioa 

LI. BackgroBBd 

An implementation technique for functional languages tkat hai received recent attention is 
reduction. In reduction machines, the prc^am is represented as a directed graph of operators and 
data, and is executed by the repeated applicati<m <rf identities, or rctfwrf Ion niUs, that simplify por- 
tions of the graph until the original graph is transformed into the final result. Reduction machines 
can be divided into two broad categories: string reduction machines, in which there is no sharing of 
subgraphs, and graph reduction machines, in which there may be. The subgraph sharing in the 
latter can confer self-optimization properties upon its programs; the G-machine' and the SKIM 
machine' are uniprocessor machines that attempt to exploit this property. 

Both graph reduction and string reductiini approaches offer cqiportunities for parallel evalua- 
tion unce several portumt of the program graph may be reduced simultaneously. Mago' has 
described a parallel string reduction machine; Keller et. at*, Darlington and Reeve', and Seep and 
Burton', have each made proposals for parallel graph reibictiim machines. The proposed graph 
reduction machines use different reduction languages, different communication networks, and dif- 
ferent mechanisms for coordinating parallel executira. mddng it difficult to compare the 
machines to determine what aspccU represent necessary features of all graph reduction machines 
and what aspecta are features oi the individual machines. 

t2. Parallel Graph Redaction Machines - A Systematic Approach 

Figure 1 depicU the hierarchy oi inues relating to the design of a parallel graph reduction 
machine. At the innermost level is the reducticm base la^uage itself; that is, the set of rules for 
transforming a graph into a printable answer, aloig with an algorithm tot their systematic applica- 
tion. Since the design of a sequential reductirai machine sodi as the ^machine encounters these 
issues alone, the issues at this level can be called the sequential-iemantie issues. 
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Figure 1. Hierarchy oi. Issues in the Design ot a Parallel (kaph Reduction Machine 



One level out are the issues related to the 'parallelizatiott' of the reduction process. Any 
parallel reduction machine attempts to employ many indivi(faal processing elements (PEs) in the 
concurrent reduction of a ringle grq>h. This introcbces probtems of where to place the graph in 
relatitm to the PEs, erf what information must be communicated by the PEs, and of what work 
must be done by each PE over and above the applicatioa of reduction rules. These can be caUed 
paraUelsemantic issues. 



Finally, at the outermoft level, it the ttnictiire at the comnmiiicatioiu oetwork that rapports 
the intra-PE inf onnati(» flow protcribed by the paraltel Kmaatics; this level is calted the toptdogi- 
cal level. As will be seen, the issues related to load balaneiag are msM appropriately dealt with at 
this level. 

Past proposals for parallel graph reduction machines have made no attempt to discuss the 
israes in each of the three layers separately. In partfeular, the boundary between the sequential- 
semantic and parallel-semantic layers is mually blurred, obscuring the distincticm between 
language particulars and essential parallel reductkm mechaniim. No authm has yet given a com- 
plete and detailed description ot all issues embodied in the puralletsemantic layer, yet it is pre- 
cisely these issues that are the essence at parallel gn^th redoctioa. 

This paper attempts to ctmcretely de&ie utA describe those aq>ects ai a parallel graph 
reduction machine that fall into the pardlel-semantic level of Figure 1 in a manner applicable to 
all languages and netwmrk top(d<^ies. What emer^s can be ttought of as an abstract parallel 
graph reduction machine, which wIku imbued with a particular reducticm language and cir- 
cunucribed by a particular comraunicaticm networic becomes a ccwrect dengn for an actual 
machine. While a language based on Tur^r^ combinatocs^ wffl be used for illustrative purposes, 
it will be Mhown that the parallel-semantic layers td tkc tadtOa^ ptopouiM, to the extent that they 
are described at all, fit the model devel^ed h^e. This in tmra mggests that all parallel p'aph 
reduction machines miut function m described Iwre at the parallel-semantic level, regardless ot 
their sequential-semantic and top<d<^ieal dedgn. 

2. The Scqaential-Scmaatlc Layer 

In order to understand parallel reduction, it te first ncccmary to underttand sequential reduc- 
tion, and so a brief ioak will be taken at Ae sequentid-seouu^e layer b^ore proceeding on to the 
parallel-semantic layer. A subset of Tumorli coaWnator temuage wOl be used to highlight the 
impivtant pmnts. 



In all graph reduction machinet, the proigram ii ezprened in a constant implicative form 
(CAF) language, in which there are no variables, only cmutants. These constanu aj^ar in a 
graph structure, and the reduction rules guide the machine in soccenively replacing substructures 
with umpler ones until all that remains is a dngle prntable result. The program graph, then, is a 
collection at nodes, where each node contains one or more lelds ctmtaining pointers to atomic 
constanu or to other nodes. When a subgraph is to be tedaeci, a piunter to the root node of the 
subgraph is passed to a reduction algorithm procedure. This procedure examines the subgraph and 
applies the appropriate reduction rules, possibly causing the reAwtion of other subgraphs or the 
creation of new nodes. When reduction is complete, the redaction procedure returns the value 
that resulu. and replaces the cmginal contents of the root node ot the subgraph reduced with the 
result of reduction. The three important characteristics et the reductitm algorithm are: 

(1) It is a procedure that takes one argument: a pmnter to the root node of the subgraph to be 
reduced. 

(2) It returns one value: the result of reducing that subgraph. The result may be an atom or a 
more complex value. 

(3) It has the side-rffect of modifying the grq)h. The most important side-effect is that the root 
node at the subgraph reduced is replaced with the result of reducticm. 

Because the root node <rf a subgraph plays swh an inqKnrtant rde in that subgraphli reduc- 
tion (iu address is passed to the reduction procedure; its coatentt are replaced by the result), 
'reducing node N* is considered synonymous with 'reducing the subgraph of which node N is the 
roof. 

To get a feel tat what kind of qjieraticms are invcrived in the reduction of a node, a language 
based on a subset oi Tumer% combinator language wiU be presented. While Tumer% combinatm 
cfKie is perhaps the least readable tA all CAF languages, tU seasiratics are quite simple and elegant, 
allowing the essential features of all CAF languages to be hij^^ted without getting too bogged 
down in language details. 



The reduction rulea tat a subset ai Turner^ iBogoage is Aown in Hgure 2. In that figure, 
lowercaie lettera refer to any arbitrary graph, the n<rtati(m <x> means 'the result ot reducing x*, 
and the left arrow indicates both what is returned and what rqdaces the node being reduced*. Fig- 
ure 3 shows in detail the reduction procedure to apply those roles. Here are stnne examples 6t 
reduction using this procedure; it win be helpful to refer to Figure 3 when reading these examples. 

Example 1: C » I 4-. 

Suy 1: kt r -Re(kKe(fB(ff» -I 

An MOB k aitdif rednced, bjr deftridoa. 
Sup2: Mfi-Redw9e(op(S9- + 
Stqi 3: WtHe<ip(fi0 

The grq* k kf t at I +. 

•ndtheato* -l-kreiwasd. 



To compute </ x>, 

use the tallowing rules to compute «^>x>: 

<lz>-<r> 

<Kx>->Kjr 

<K*y>-»<r> 

<+at>-. +x 

<+jr y> - <r>+<y> 

<3/>-.S/ 

<8f g>^Sf g 

<Sf gx>^<f x{gx)> 

(rtherwise, - ElUKHt 

Figure 2. A Small Reducti(» Lai^age Baaed <» Turner^ OMnbinatmi 



*U the rcMk of redactka k ■■ Mom m, by oomrcMka the node tedaeed k raptaeed by I «. Soch • wide k caffled an la- 
Mncila* medt by Iteaer. 



The Reduction Procedure: 

Given a pmnter to ■ graph, £, reduce 

the graph and return the result. 



Redwe(E){ 
SUtt: 
Mr-Rednce(fD(E)): 

ttT'lthmi 

Mfi-IledKs(ap(E)); 

Writeop(E.fi): 

ntnnifi;} 
•Itiirr-KthM 

write-fii(E.r): 

Ntaraf; 

Wrile-fii(e.r); 
iitua Bi 

•taiirr-sthM 

Write-fii(S,r): 
nttnuBi 



/••I»er«le<l *>-.<![> 7 



/*T1ierale<Xjr>«Kjt7 



/•The nrte <+*>-+ sir 



/*-nerak<3/>-S/ 7 



/•The'emxniei 



Write-fB(£JI^ 

Writo«p(e.KUt<«); 

ntuvUtKOB:}} 
■te If f B(r) k u MOV Am { 
irfii(n-Kthai{ 

tot fi « Ite(tooe(op(r)); 

Write-fB(ff4); 

Write-op(E.0; 

NtmO;} 
•totf fB(7)-+tkn{ 

tot 'Re(tace(ap(7^ +RedDcs(ap(E)): 

Write-fB(£4): 

WriteKspCCO): 

»*twt«C;} 

•biirfB(r)<=8 

Write4B(ff.7); 
ntanC;} 
•tattf fB(fB(r))to 

irfB(faOT)-BthM{ 

totF-aiKfnCD): 

totC-ap(l): 

ta(X-a|>(E): 

writ&4B(e.CK>te(r;r)): 

Write«p(e.Create(Cjr)]e 
■•to Suit;} 
} /* EmI of procedarc Redaoe 7 



/* The nfa <X z jr> - <r> 7 



rtbe rate <+* 7> - <«>+^> 7 



/■tlw rate <S/ «> - S/ f 7 



/* Hie rate <8/ f x> - </ X (t ji)> 7 



tkktr Pnetdmm CaOti kf KedMe 


fB^) 


RetwBe the fBOctioa IcU of tte Mde pofated to bjr £. 


0P(B) 


RetwM tke opoud Md of Mto aod* ffOfaied to by £. 


wiit»fi<Bjr) 


Wttoi X ia ttM fHHltoa Mi af Am aadt foiBtod to bjr £. 


Write4ip(Bjr) 


Wttoi y fa tte aptiBad >■!! af Mw aadi faiaied to by g. 


CreateCrj) 


Oeatae a Bear aade. UHUtoM III ftHMioB Md to X 
and to opmad teld la r, aad nl^ae a fatater to it. 



Figure 3. A Reduction Procedure tat the Language in Figure 2. 



M 



Example 2: £ = (I 4) 3. 

sup 1: tat r - Redaoc(f n(E)) - + 

TUt radMcdoa w« ilhHtrtf ed fai Eu^ile L 
Stqi2: WrkMip^.!) 

Tke IT Vk b Mt IB 4- 1. 
Stq>3: ntsnV 

■nd + 3 k retuned. 

Example 3: £ = ((I 4) 3) ((+ 4) 5) 

Supi: tatr''IUdacc(fit(E^-+3 

This nedBcliaa wm UMtnoed ta ExMqde 2. 
Step 2: tatO-ReaMseCapOf^-t-RedaceCapCE^-a-fV-B 

opCO » 3 (■■ mob), and op^ - (+ 4) S, wUck mlHCS to 9. 
Step 3: Wiite4i^J0 
Step 4: Wift»<ip(K,0 

The ^qpk k lef t u 1 23. 
Stqi S: ntm fi 

■ndtkeatoH 12 b fctwaed. 

Example 4: £ = ((S 4) (+ 3)) 4 

St^ 1: tat r -Redw«(fB^} -CI +) (-(-^ 

Thta radaetioa b riidir to Etuqile 2. 
Step 2: tatF-ap(fi^n>- + 

fB(n -• +.10 op(f^l% - +. 
819 3: tatC-ap(r)-(+9 
Step4: tatX •'Op(E)-4 
Step S: Writ»faCE.GKate(Fjr^ 

ni f a b aow ilw ae« gn^k + 4 
Step 6: Wtit»«p(C,GKBte(CjrA 

£^ op b aoir tlw new 1^^ (•«■ 3) 4 

Heacc. S b aoir A« gnpfc (+ 4) ((+ 9 4) 
Stqi7: 1^0 Stat 

The lAolc Mdaetioa praeedate b Halted ^ida oa tka aew venioa of ff. 

IWe a4i cveatadiy lit ledaeed t« U. 

These four examples are tjrpical oi the Qrpes <tf rettactkn roles encountered in most reduc- 
ticm languages. In Example 1 the node is uiwlianged; w Bxwbb^ 2 acMne descendents of the node 
are reduced and the rewltt stored back into dte node; ia EzsBRf le 3 descradenu are reduced, a 
computation perf cvmed on the results, and the resist of the eo^^tation returned and sttved back 
into the graph; in Example 4 new nodes are created, the grspli i«iffranged, and the reduction rules 
reapplied to the result. It slKHild be a<Ked that in Exa^de 4 the node is is ctnsidered reduced not 
at Step 7 but mily wlwn a r^ani statement is bia^ raueotod; the writing of a node does not 
necessarily take place tmly at the condusicm of itt re <fac ti oB. it should also be noted that in Step 
2 of Example 3 the two reductions requited could be |wtfonBed rirauitaaemtsly in a parallel 
machine; in general parallelinn is obtained by 'forkuig* dema^ across strict curators in this way. 



u 

While there are many CAP languagoi other than Tomer'k, the reduction procedures to 
implement thow languages will be (juite similar to the procedure in Figure 3. A careful examina- 
tion of Kgure 3 and the examples presented wUl reveal that there are onty five kinds ci opera- 
tions performed on the graph during the reduction of a node N: 

(1) Reading the fields oi node N. 

(2) Writing the fields of node N. 

(3) Creating new nodes. 

(4) Calling tot the reduction of descendent nodes c^ node N. 

(5) Reading the fields of those descendent nodes that have been reduced. 

(The term "<fescendent nocte <rf node N" here denies a node that is reached through the trac- 
ing of a chain (rf p<Hnters ci bounded length rooted at node N^ It is particularly important to note 
that the tmly node an instance of the reAicticm procedine writes is the node it is reducing. Stated 
another way, a node can only be altered by the instance of the reduction procedure that re<faiccs 
it. This implies that once a node is reduced, U It mev& writtem a^^; nodes become constanU after 
they are reduced. 

The five kinds ot operations listed above are the only ways ki which the reduction procedure 
is permitted to interact wiUi the pri^am ffnph. Any oth^ OM^niUtkm performed by the reduc- 
tion procedure is limited to manipulatkm of its internal sti^. Snc^ manipulati<Hi would include 
arithmetic operatitms on data obtained fr<nn the grafdi, comiMrisaiu in order to select a reduction 
rule, etc. Limiting the reductitm proce<hire% access to the gr^ to the five operations above is 
not an arbitrary rMtrictimi but an obaervatitni thM reflects the nMure oi graph reduction in gen- 
eral. This universal property of the sequentiat-aemantic layer wOl be the guiding fcvce in the 
devek^ment of the parallel-semantic layer. 
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3. Tbc Parallcl-ScBaatic Layer 



3X Machine OrganbatloB 

In a parallel redaction machine, there are many processing elements (PEs) all trying to 
reduce one graph. The irtt question to be resolved, then, is where the graph is to lie in relatiim to 
the PEs. An obvious approach is to place the grqih in a nwaMiry that is tkar&d among the PEs so 
that each PE has equal access to aU nodes Of the pi^. White this approach is conceptually 
attractive, it introduces severe problems related to maintaining atomicity of operations performed 
upon the memmy. Furtherm<ve, it is dear that contenti<» for the lAared memory will swamp the 
benefits obtained from parallelism for even a modnt Burab«r of FB». 

To eliminate the ctmtention issue, each PE is given a CMtaia amcmnt ai ita own local graph 
memory, to which only it has access. This in turn requires thm tte program graph be t&tributed 
among the graph memmies of the PEs, and so iKMfcs of the gtwfk most be able to pcnnt to other 
nodes that reside both in the local PE and in etbet PBm. A paia^tet to a node, therefore, must be a 
tuple of the form (P£ mdJress), where P£ is dw PE <m fi^leh Ae node p<»nted to resides, and 
addreu is the acklress in that PE^ local memory. AnoCl^r wqr <tf viewing this scheme is as one 
large contiguous address qiace that is divided s|> unoBg the PCs. A node resi^ng in the memory 
of one PE can r^tr to a node residing in a ^ferent PE, but ■ node can be read ot written cndy 
by that node% PE; i£,by Uie PE in whose load memocy thitt aode reddes. 

Of course, there must be some uxX. oi ammmdeatiom metwtrk between the PEs if they are to 
work in ccmcert. In designing the paraUel-araiaatic Iqrer Oe oBty aasumptini made about the 
communications netwcvk n diat a PE may arad an arbitrary BMssage to another PE; all <Mher 
details (rf the netwcvk are properly dealt with in the topokisiad layer. While the communicati<m 
network is in some seue a Aared restraree. the dei^ at the U^^igteal layer can be chosen to 
reduce any contentic« i»oblema to a suitabte level; the saase caanot be said for a shared memory. 

Distributing the nodes anumg the local memories oi the PEs provide a natural way to divide 
the w<wk (A reducing the grs|>h: the watk of re<tecing ^y pwticolar node - applying reducti<m 



nilei, etc. - is assigned to that nodeli PE. Node (2 45). therefcve, will always be reduced by PE 
number 2, node (7 12) by PE number 7. This asngnment ol wotk bt mily natural, lot the reduction 
of a node N is guaranteed to require reading and writing the ftdds of node N. and only node N^ 
PE has the privilege at accesung node N. One effect ct thto avipunent is that the distribution of 
nodes among the PE^ memories is equivalent to <fotributing wodk am<mg the PEI processors; if all 
nodra of a graph were placed in one PEIi memtny, only that VB^t processor could take part in the 
reduction of that graph. 

32. Intcr-PE Cemnwnkation EsscBtfads 

With the basic structure of the machine in hand, it Is now necessary to make it function. In 
the previous section, the five kinds ai operation performed on a i^aph during reduction were 
enumerated. It is the task of the parallel-«emanttc tay^ to iumxe that a method for accomplishing 
each of these operations exists in the parallel niKh^. 

Implmenting the first two operations, reading and writing the node being reduced, are easy, 
since the node being re(hiced always resides in die graph memory of the PE performing the reduc- 
tion. These operations are simple accesses to local memory. 

The third and fourth kinds ai operations, creating new nodes and calling fw the reducticm at 
existing nodes, require the asnstance of onhet PEs; tlM fonaer because new nodes will sometimes 
have to be created mi other PEs to (fistribute die workload, wad Oe latter because reduction of 
existing nodes is ccmstrained to take place <m each ta^vidoal aode^ PE. In a sequential machine, 
the reduction procedure would accomplish these opnatiOBS tfao^ procedure calls: a call to the 
'create* procedure creates a new node and returns a pointer, • call to the 'reduce' procedure 
reduces a node and returns the result. In a sciential ou^iiw, of course, the latter is a recursive 
call. The reducticm procedure in the parallel madhdse niso ean accimplish these operations 
through procedure calls, but in this case these procedures mi^ require executi<« on a different 
PE. What is needed is a remote procedure catt f Kility. 
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To implement remote procedure calls, we turn to the communicationi network. A remote 
procedure call in the parallel reduction machine is accomplished by a pair of messages: a request 
message, sent from caller to callee, communicating the arguments of the procedure, and an ack- 
nowledgement message, sent from callee to caller, communicating the results. Any side-effects 
caused by the remote procedure are restricted to the local memory of the callee. A request mes- 
sage takes the form: 



reqesl-id typ e-REQ argwnents ] 



while an acknowledgement looks like: 



reqestAd j fypg-ACK results \ 



The type fields of the messages indicate in effect what procedure is being called, and the request-id 
field, copied by the called PE from request to acknowledgement, allows the acknowledgement mes- 
sage to be routed to the calling PE and identified there. Figure 4 lists the messages used in paral- 
lel reduction. 

The first two messages in Figure 4 are used in the creation of new nodes. Suppose PE #1 
wants to create a node and have it reside in the memory of PE #2. From a semantic point of 
view, PE #1 would like to call a procedure like Crtmtt(initial-contents), where initittl-contents are 
the initial values for the fields of the new node, and have a pointer to the new node returned as a 
result. Note that PE #1 expects not only a returned result, but also the side effect of the creation 
of a new node. Using the remote procedure call mechanism, PE #1 prepares a CREATE-REQ 
message and sends it to PE #2. PE #1 then waits until it receives a CREATE-ACIK message whose 
requested field matches the request-id it created for the earlier request. When that message is 
received, PE #1 examines the results field to obtain a pointer to the new node. 

From PE #2*8 point of view, PE #2 receives a CREATE-REQ message. It responds by allo- 
cating space for a node in its local memory, initializing the new node according to the initial- 
contents field of the message, and sending back a CREATE-ACK message containing a pointer to 
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(1) Crcatfcm RcqacH 

\ requeu-id \ CBEATK^tEQ \ b^iai-amientt j 
ReqaoU the creatiaa of a new node ioiliaHaed lo / 



(2) Creatim Acknowledgeaitirt 

I reqmtt-U \ CEgAl»-ACK [ lUw-polHter \ 
Infonu the •ender of a fXKATUtBQ aiwiagp that dw Mw ■odcfapnilerlto bjr mtw-^elmim 



(3) Rcdactioa Reqacit 

I requested \ RBPUCE4tBQ { poUaer | 
Reqaeati that the mAg^afh painted to b; ppb^tr be i 



(4) Redaction AckaowledgeawBl 

I request-id \ RBMJCaB-ACat | ruidt] 
laforau the WBder el a UDDCX-MBQ BeMafB that the taaalt of n adMl i eM ii ran*. 



(5) IiKrcmeiit Refcrciicc Ceaat Reqacfl 

I requeMt-id \ INCRiy-RBQ | pointer 
Reqoetfa the refereooe ooutt of the node pointed to by palaMr be 



(6) laercmcat Refcrcacc CmwI Ackaewlc^pawBt 

I reqmtt4d \ PICRgr^CK | 

Inf oTBi the tender of an INOtKr-UQ atiMMpi Aat the nfcMBoe eoHt hm been ineKBented. 



(7) Occremcat Rcfcreacc Court Roqaol 

I reqmst-id | PRCRRT-RBQ | poiiuer j 
ReqMMs the ref ereaoe ooMt of the node potated M by jp«ia*ar be dtocnaMMd. 

All mevaget cany a reqaeM kfcotiicatx» ia 1^ §M ntpm»'U. The request identification is 
created by the issuer of a tcppuBHt. and oqried f ran tcf^at lestage to aduuni^edlgement message 
by the receiver id a reqaest. 

Figure 4. Inter-Processor Messages 



the node. The pointer, of course, will be (tf the farm (2 mddresg). The req»ett4d field of the 
request message contains the name of the sender, ¥B #1, to Att PE WT. knows to whom to addren 
the acknowledgement. PE it2 copies the entke re^mt-id field from revest message to ack- 
nowledgement. Thus with the aid of the firrt two messages in Figure 4. the third kind of operation 
required by reduction algcnithms is acoMsiodated. 

The next two messages in the Figure implement the fourth kind of operation, the caUing for 
of the reduction of another node. Here, the proeeduK caH ^aviated is Rcdacc(politfer), where 
poiiatr u a pointer to the node to be rethiced, which returns tte result ol reduction as well as hav- 
ing the side effect al altering the node reduced. The tasploMntatiai ci. this procedure through 
message passing is analogous to the implementaticm of the 'neate' procedure: a RBDUCE-RBQ 
message carries a pmnter to tte node to be reduced to that iradeli PE, and that PE reqKmds by 
reducing the node and sen<fing back a REMJCX^CX message Aat contains a copy <tf the result. 

The subject of what exactly is returned in a KSMKX^CK message requires some thought. 
If the result of a reduction is an atcnn, then the atom itself earn dmiply be returned. If the result 
of reduction is a subgraph, however, it is not obvious what most be returned. Merely returning a 
pointer to the subgraph is not always sufficient, for the eall« wffl generally need to access some 
tA the nodes in this subgraph 0«i the fifth kind til <q>eration as Inted in Section 2), which it can- 
not do if the subgraph remains on ancrther PE. On the other hand, the entire nibgraph should not 
be returned, mA only beca^ this is far more inf ormatkm thu k needed, but also because the 
entire subgraph is not necessarily available to the PE inepuing the acknowledgement, as it may 
be distributed across many madiines. 

The rimplest pcdicy is to return a copy of the root node of the subgraph to be returned; that 
is, to return a copy vl the node redwed. Ite PE reoetvi^ ^ acknowle(^ement then takes the 
node fr<nn the acknowle<^ement ud plues it in ita own kml meniOTy, and may then treat the 
new node in local memory as thou^ it were the trade on Ae fwiegn machine. In dtHug this 
operaticm, two copies of the same node are ne^ed, rmfaig the question of consistency. There is 
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no need to worry about consistency, however, for the node cq)ied is a node that has already been 
reduced. As pointed out in Section 2, a node that has been redaced can never be dtered again - it 
» effectively a constant until it is garbage collected. Thm, oeating a copy €l a reduced node is 
safe, since it amounts to creating a copy tii a ccmstant. 

Before moving on, it is worthwhile to consider an example. Figure Sa shows the program -f 
(* 3 4) 8 distributed Kross three PEs. He nxtt node is at address mi PE #1. the twonode 
expression (* 3 4) is at addresses and 1 on PE #3, and the remaining node is at address on PE 
#2. The reduction at the pn^am bcjpns with the f(^owing aacasa^ sent to PE #1: 

I requeitAd \ KBDUCB4UIQ \ (t^l 

PE #1 starts to apply the reduction procedure shown in Figure 3 to the node, whose first step is let 
T == Reduce(f n(£)). f n(e) is the node (2 0), so PE M sends Uie ftrilowing message to PE #2: 

I request-Id \ RBDUCE-IBQ \ (2 0)] 

PE #2 responds by applying the reduction procedure to node (^ 0), and fin<b that since the func- 
tion is the atom 4-, the node should be returned unaltered. So re #2 sends a copy of node (2 0) 
back to PE #1 Uke so: 

I requeit4d \ RBPUCK-ACK | [(ATCail 4) (3 0)] | 

When PE #1 receives this message, it creates a node in its own memory and puU the copy tA(^Q) 
there. At this point, the PEs' memcnries appear as in Figure Sb (the function p<Hnter of node (1 0) 
has not been changed f nmi (2 0) to (1 1), as might be expected, but the pcnnter to (1 1) is kept in 
the temporary variable T oi the re<hictifm procedure cxecutii^ on VE #1). The reducti<m pro- 
cedure on PE #1 now resumes, and sees that Uie statemoit If f ^T) * + is satined, and proceeds to 
call tot the reductions <rf the operands <rf nodes (1 0) and (1 1). Node (1 0)11 operand is an atom, but 
node (1 1)% operand is the graph at (3 0), which is reduced by semBng a reduction request to PE 
#3. PE #3 responds with a reduction acknowledgement eoataining the atom 12, and PE #1 
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reduces node (1 0) to I 20, sending a recfaietioB ■cknowledkemrat ccmtaining the atom 20. Figure 
Sc shows the final qypearence ot the VBm' aiemories. 

In the example above, the result of rechu^g node (2 0) mm the three node subgraph -I- (* 3 
4), but it was sufficient for PE #Q to return <mly tbie root nf>de to PE #1 in the reduction ack- 
nowledgement, for the rocrt node contained tSk informatiott needed by PE #t Consider now the 
reducticm erf 8/ g x, where each ci the three nodes are on different PEs as shown in Figure 6a. 
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Reduction begins on PE #1, which sends a redDcti<n nqaeat to PE #2, which in turn sends a 
reduction request to PE #3. PE #3, seeing that the f unction is the atom S, sends the foUowing 
acknowledgement to PE #2: 



request-id \ REPUO^ACat | {(ATOM S) (/M1 

PE ilQ, copies this node into its own memory, and the menKwies are now as shown in Figure 6b. 
The reduction procedure on PE #2 sees that the statement IT f nCT) => S succeeds, and so wanto to 
return the twanode result (S/) «. If only the root node ot a graph b returned, PE tlQ, sends this 
message to PE #1: 

I request4d \ RBPUCaB-AOL | [(2 1) (y)] | 

When PE #1 receives this message, it will have two erf the three nodes comprising the S expres- 
sion, but to apply the reducticm rule for S it needs all three, for it needs the pmnters \of,g, and x 
(in fact, at this point it is miaung the node that contains the 8$. In this case, PE #2 must actually 
send two nodes back to PE #1, both of which will get copied into PE #1% local memory. This 
would be accomplished by a message like this: 



request-id \ REPUCB-ACK | {[(MSO 2) (g)] [(ATOM S)^^ 



In this message, the pointer (MSG 2) ptrintt to the secimd node ctntained in the message; when PE 
#1 copies the contenU of the message into iu own graph meaocy, it will replace the (MSG 2) 
pcnnter with a pointer to the actual node created for the secMMl node in the message. Figure 6c 
shows the state oi the memories after PE #1 inktes thui copyteg. 

When a graph is to be returned from reduction, then, the rule for determining which nodes 
to include in the rechicticm acknowle^ment is as fcdlows. The root node of the graph to be 
returned is always included. In addition, any nodes pmnted to by the rotrt node that were returned 
from reductions requested during the redoctira ci the root mide are also included. The nodes in 
this set are known to be reduced, making it safe to send them hi a message, and are guaranteed to 
be accessible to the PE creating the acknowledgement. 
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33. Tkc Need for Moltl-TaddBg 

In the preceeding diicuwioo, no mention wu made of what • PE must do if it receives addi- 
tional requests before dispensing with the one in progress. When a PE processes a reduction 
request, at several points it will send requests of its om ^k1 wait tot the corresponding a^- 
nowledgemenU. It is unacceptable for the PE to suq>end dl activity when waiting for ack- 
nowledgements, because the requests it makes may cmse other PEs to send adtUtional requestt 
back. If the PE ignores those requests, it will never receive the acknowledgements it is waiting 
for, and a deadlock occurs. Because the processing of a ttdac^on request may be suspended while 
waiting tot service frmn another machine, a ¥E mvM be capaUe of procesung several reduction 
requests at once. 

A single PE, theref<n-e, can have several outstanding reduction procesies, each one 
corresponding to a node currently undergtnng reductimi. Associated with each reduction process 
is a process descriptor (PD), which has enough infmrnation to aOow the process to be suq>ended 
while waiting for acknowledgements and later resumed at Hbs point of suq>ension. A process can 
be in one of two states: suspended or runnable. A wsagpenadeA ^vcess is one that has sent requests 
but has not yet received all c(vresp<Hiding acknowledgMaents,«Kl a runnable process is either one 
that has just been created or one that has received tXL a^iunriedgeraents. A runnable process will 
be selected by the PE lot execution, at which potet the redac&m procedure will be resumed on 
that process until either one or more reque^ are isnied, cawrisqi die process to become suspended, 
or until the algmithm inishes, causing a re<hiction acknowIed||aBawnt to be sent. A suspended pro- 
cess becomes runnable again when it receives dl acknowiedlgaBente tot which it was waiting. Fig- 
ure 7 illustrates the states a process can assume. 

When a particular process^ instance oi the redncti<» procedure wants to make a request, it 
must do two things: it must send the ajqnopriate request messages, and it must indicate in the pro- 
cen descriptor that it is waiting tot acknowledgements. The V& may then pick another runnable 
process and work oa'ixttx n while. When Kknowle<^ment messages are received, they must find 
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their way to the correct process descriptor and return the process to the runnable state. To organ- 
ize the flow of information, each process is assigned a unique ^ocess number, and several request 
slots are provided in each process descriptw. Recall that messages always contain a request 
indentifier. Whenever a process sends a requMt message, it uicludes a request identifier of the 
form (PE process slot), where PE is the number anigned to the revesting PE, process is the pro- 
cess number of the process making the request, and xfoi is the number of a request slot in that 
process descriptor. After sending the requeM message, the process stores the atom WATTINO in 
request slot slot at the process descriptcv; any process descriptor that has the atom WAITING in 
one or more of its request slots is ccmsidered suq>ended. Any acknowledgement arriving at the PE 
is stored in slot sUa of process descriptcv process, where slot and process are taken from the 
request identifier of the acknowlec^ment (remember that the request identifiers in acknowledge- 
ments are copies of the request identifiers contained in the ctvreqiondings requests). When a pro- 
ceu receives the last acknowledgement it is waiting for, that acknowledgement replaces the last 
occurence of the atom WAITING in that process^ requert slots, and the process is considered 
runnable. When the reduction procedure is resumed on that process, it can find the results it 
requested in the request slots, for that is where the acknowledgment messages are stored. Note 
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that • proceu can make several requetts at oacc by aendiBg aeveral request messages, each with a 
different value of slot in their request identifiers; this is how parallelism is achieved. 

Another function ci the process descr^tor is to Irald the re^test identifier d the reduction 
request message that created that process, tot that infrnmatioa is necessary when preparing the 
reduction acknowIe<^ement when the reduction procedure terminates. Because of subgraph shar- 
ing, it is possible for a seccmd revest to reduce a given node to antve while the first request is 
still being processed. It is not safe for a sectmd process to be started on that node, because the 
two processes will interfere with each other. Instead, only (»e process is allowed to reduce one 
node, but a process is allowed to send any number oi reitectioa acknowledgements when it com- 
pletes. To keep track of this, the process descriptcv will contain a list erf iiotffiert, one for each 
reduction request received tot the node being reduced by that proceu. A notifier is merely the 
request identifier frmn a reducti<m requort menage; when the process completes, one reduction 
acknowledgement will be sent tor every notifier in the notifier Ust, and the requested fields ot 
these acknowledgements will be created from the informatioo in the ncrtifiers. 

Support for multiple processes also requires additiond infcMmation to be stored with each 
node. Each node must have, in additicm to the data fields jmiat^bed by the sequential-semantic 
layer, a itatus field. A node can be in cme of three states: imreduced, reducing, and reduced. 
When a node is created, either thrcnigh the procesraig <rf a GRBATB4tEQ message or through the 
copying ai nodes received in a REDUCE-ACK messi^, the Matus field is set to UNREDUCED. 
When the first reduction requett to reduce that iKxie urives, a process desoriptcw is created and 
initialized, and the process descriptcv number is stored in the ttatus field ol that node. Thus, the 
presence of a process descriptor number in the status field of a node indicates that the node is in 
the 'reducing* state. If additi<mal requests to reihice that node anive while the node is in the 
'reducing* state, the statin field ot the node intfieates which process descriptw should receive the 
additional notifier. When the process finally finidha redodng the node, the status field of the node 
is changed to REDUCED. Servicing any additi<mal requests for die reduction ot that node will 
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timply entail reading the node and preparing the appropriate reduction acknowledgement. Ai was 
noted earlier, once a node enten the REDUCED atate it effectively becomes a ccautant. 

3A. RcfcrcBcc CoBBt Garb^tc CoilectiM 

Because of the dynamic nature ot reduction graphs, garbage collection is an important con- 
cern in the design of a graph reduction machine. It is doubly important in the parallel graph 
reduction machine because of the copying of nodn from (me VB to another when reduction ack- 
nowledgements are sent. A useful propoerty of most reducticm languages is that they can be 
defined in such a way so as never to create cyclic graphs. Turner^ language, for example, can be 
made to either create cyclic graphs or not create cyclic graphs depending <m the implementation 
of the Y combinator. In general, the avoidance of cyclic gr^hs entails a small amount of addi- 
tional work during reduction, but there is a potentially great savings in the time required fm gar- 
bage collection, for in the absence (tf cyclic grapbi r^ermc* emmt garbage coUeetloH can be per- 
formed. 

The mechanism necessary for reference count garbage collection is easily added to the sys- 
tem already <tescribed. Each node in graph memory is augmented with a r^erence count field, 
which is initialized to tme when a node is created. When a rettactioB process creates an additional 
pointer to a node, it sends an Increment Reference Count ReqoeA (INCREF-REQ) message to 
that nodeli PE which ccmtains a pointer to that node. The PE receiving an INCREF-REQ message 
responds by simply incrementing the reference ccwnt of ttat node. Similarly, when a node des- 
troys a pointer to a node, it sends a Decrement Reference Count Request CDECREF-REQ) to the 
nodeli PE, which responds by decrementing the reference count of that node. If the reference 
count of a node is decremented to zero, IMBCREF-REQs are issued to the PEs of any nodes 
pointed to by that node, and the node is returned to the free list. 

Since INCREF-REQs and DECREF-RBQs can be issued tot a given node by several PEs at 
once, precautions must be taken to make sure that these messages do not arrive out of order. If 
the reference count (tf a node is one, for example, and an INCREF-REQ followed by a DECREF- 
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REQ ii issued for that node, if the messages arrive out of <wder the reference count will drop to 
zero before the INCREF-itEQ message arrives, and the node will be garbage collected even 
though a pointer still exisu to it. To prevent this occurence, it is noted that any time a process 
creates a new pointer to a node, it must already have a p<rinter to that node. Even if the 
INCREF-REQ message never arrives, the node will not be garbage collected as long as that pro- 
cess retains the origind pmnter it had to that node. Thus, the process issuing an INCREF4tEQ 
can guarantee the correctness ol the node'k reference count by tuspemfing its activity until it is 
sure the INCREF-REQ message has been received. 

The obvious way to accomplish this synchrcmization is to have the issuer of an INCREF-REQ 
enter the suspended state until it receives an Increment Reference Count Acknowledgement 
(INCREF-ACX) message, which the receiver ct an IN<SBF4UEQ sends after incrementing the 
reference count. In this way, the process canned accidentally tame a DECREF4tEQ for that node 
until the INCREF-REQ has definitely been processed, and so the reference count will never be an 
underestimate. There is no need to have a Decrement Reference Count Acknowledgement, for 
there is no danger in overstating the reference count tempwarUy. The issuer of a DECREF-REQ 
can proceed immediately after issuing the message. 

33. Smmmmrj 

The essential design of the parallel-semantic layer is coiBiriete, and is now summarized. The 
overall appearence of the parallel reduction machine is as illuMrated in Figure 8, with a number 
erf identical Processing Elements c<mnected by a communicati<»s network. The communications 
network is of arbitrary topology, but must nippoct tlw reliable trannntssion of messages from one 
PE to another. 

The flow of information within each PE is dqiicted in Figure 9. There are two types of data 
stored in the menuny of a PE: nodes and process deseripton. Nodes, which are the (Ejects 
comprising the program graph, are stwed in Graph Memmy (GM), and ccmtain, in addition to the 
fields prescribed by the sequential semantic layer of the particular machine, a status field and a 
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reference count field. Process Descripton keep track (tf the tasks in progress within a PE; there is 
one active process descriptor for every node in Graph Memory diat is in the 'reducing* state. The 
process decriptor ctmtains a list of notifiers, one for every RIDUCB-ACX message that will be 
sent upon the completion of that process, a set of request dcrts used both to indicate the status 6t 
the process and to hold acknowledgements after they are received, and enough state information 
to resume the reduction procedure after it becomes suqiended thrmigh the issuing ai requests. 

There are logically three distinct computational entities within each PE. The borage Mes- 
sage Processor handles the processing oi incoming OtBATB-KEQ, DiCREF-REQ, and DBCRBF- 
REQ messages. In processing these messages, the SMP re^iim acceu to the Graph Memory, and 
can issue CREATE-ACK, INCREF-ACK, and DECKEF-RBQ messages. The latter arise when 
nodes are garbage collected, and since DBOREF-ftBQ messages have no corre^onding ack- 
nowledgement, the SMP does not need to suspend its operati(»s at any time. 

The remaining messages, REDUCE-RBQ, KBDUCE^GK, CREATE-ACK, and INCREF-ACK. 

are handled by the Computation Message ^ocMsor. The latter three menages cause the writing 
of request slots of process descriptors in the suspended state. The REOUCB-REQ message causes 
the status field of the node indicated in the message to be examined. If the status is 'unreduced*, 
an unused proems descriptw is obtained and its number Mored in the status field of the node to be 
reduced. The state inf cvmatitm in the new process descr^tOT b initialized so that it p<»nts to the 
beginning of the reduction procedure with the node as argument. Finally, the notifier list of the 
process descriptor is initialized with the re<picM-4d of the RBDUCB-REQ message. This results in 
a new runnable process. If the status field oi the node in the RBDUCB4tEQ message was already 
the number of a process descriptw, the request-id is added to the notifier list of that process 
descriptor. If the status field of the node was 'reduced*, the operatiois performed are exactly the 
same as if the status field was 'unreduced*, except that the state inf cwmation in the new process 
descript<» is initialized to beipn at the end at the re<hicti<» procedure: at the beginning <rf the 
section that sends the reduction acknowledgements and removes the PD. 



Processes move from the suspended state to the runnable state only upon the receipt of a 
message, so the Computation Message Process^ is capable (tf providing a stream of process 
descriptor numbers of processes that have moved from the suqwnded state to the runnable state. 
A PD number is added to this stream in two cases: if a REDUCB-ACK, CatEATE-ACK, Or 
INCREF-ACK is received that overwrites the last occurence ot the word WAITINO in the request 
slots, or if a REDUCE4EQ is received that creates a new procen descriptor. Hie stream of runn- 
able process numbers is passed to the Reducer, which actually perf<Hins the reduction algorithm. 
When the Reducer resumes a process, it works on that process either until it issues one or more 
requests, whereupon the process enters the suipended state by virtue of the word WAITING in 
one or more of its request slots, ot until it completes, causing ooc REDUCX-ACK message to be 
sent for every notifier in the notifier list, after which the PD is letumed to the list of free PDs. 

As Figure 9 illustrates, while the Storage Message Procesam, the Computation Message Pro- 
cessor, and the Reducer are functionally indqiendent, they diare two data structures. Graph 
Memory and Process Descriptor Mem(»y. Contention prcAlems are avoided, however, because 
their use of these structures is disjoint. The Stongt Message Proceasm', tat example, is the only 
unit that uses the free node list or the reference count fields of the nodes. The data fields of 
nodes are only used by the reducer after the SM.? creates them. The status fields of the nodes are 
used only by the Computation Message Processor. Similu dtvisitmi Ot usage occur between the 
Computation Message Processorli and the Reducer^ use of process descriptors. 

4. OptkHul Features 

In the previous secti<m, the minimum function ot the paraUel-semantic layer was described. 
There are many extensions to this basic system possible that wiU improve the performance. 

4 J. Prograai Loading and I/O 

While the capability for initial loading of program graphs is hardly an optional feature, it is 
of less importance than the actual executi<m oi prop'am graphs. Happily, providing this feature 



requires no sdditional mechanism in the parallel-semuitic layer. 

Generally, the overall machine structure aa shown in Rgure 8 will also include a special 
Front-End Processn attached to the communkation network, which can be addressed as if it were 
a regular PE. This q>ecial unit is in charge (rf all interacticm with the user, including I/O and the 
loading of programs. The Front-End Processor loads a program into the machine by issuing 
CREATE4tEQ messages, and begins its executitm by issuing a iUEDUCE4tEQ message. When it 
receives a REDUCE-ACE message, that message will c<mtain die result to be printed fcnr the user. 
The way in which I/O is handled is up to the base language, but it will usually be in the form of 
streams, whose curators interact with the Froat-Ead Processor through REDUCE- 
REQ/REDUCE-ACK message pairs. 

42. Tfanc Sharing 

Any parallel reduction machine built upon the prii^qiles set f<»rth here is capable of per- 
forming time sharing, for eadi PE already has the facility for wnking on several tasks at oaec. 
To achieve the simultaneous ezecuti<m oi two unrelated programs, the Front-End Processor simply 
loads both programs onto the PEs and sends a REOUCS4UBQ for each of the two root nodes. The 
two graphs will each get a more or less equal share of the PEs combined time, tot the PEs have no 
way of knowing that the various nodes being reduced are part of unrelated graphs. 

It is also relatively easy to provide this time diaring sjrstem with a crude pricnity mechanism. 
A priority field u added- to the process descriptw and to die RHHJCX<REQ message. When a PE 
receives a REDUCE-REQ message, it ccanpares the primrity field <rf the request with the priority 
field of the process descriptor that will process the request, and Mores the greater back into the 
proceu descriptor. Whenever a process issues a RBIHJCX-RBQ, it will take the priority field of 
the request from the priority field of the process^ process dmaipUH. Thus, the priority is pro- 
pagated to the descendant nodes of the miginal node reduced. 

The priority comes into play when the PE chcMses a mnnaUe process for execution by the 
Reducer. When the PE selects a process from the stream of mnnable processes, it always selects 
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the ninnable process with the highest pri<Mity, thus assarinf that higher priority processes are ser- 
viced first. 

43. Redaccd Idle TIaic Throogh Eager BvalBatioB 

Up to now, the parallel reduction machine has been ciHnpletely demand driven; a REDUCE- 
REQ is never issued tot a node until some reducticm process definitely nee<b the result. Some 
researchers have suggested that additional parallelism can be extracted from a program by reduc- 
ing some nodes before they are needed, so that if their values eav eventually needed they will have 
already been computed. This scheme can make use ci uiy idle time that might otherwise exist in* 
a system with a large number of PEs, but it is imp<^aBt that valuable time is not wasted reducing 
nodes whose values will never be needed. 

The priority mechanism described in the previous section provides an elegant way of control- 
ling eager evaluation. By assigning a higher priwity to the SEDUCX-REQ issued tm the root 
node of the graph than for the REDUCX4tEQs issued for other nodes of the graph, each PE will 
always work on nodes definitely needed for the ccnnputatioo of the final result if it has a choice. 
An additional problem introduced by eager evaluatitm is that nodes requiring garbage collection 
can have reduction processes active on them. The garbage coilectton mechanism must therefmv 
collect processes as well as nodes. 

44. Increased Throaghpat Through Mnttiplc Reducers 

Unlike many proposed parallel machines, the parallel redhwtkm machine described here does 
not make use of shared memory at all. (tee coisequence is that each PE must multi-task: a PE 
can have several runnable processes existing at once. The throu^tput of a PE can be improved if 
the PE in Figure 9 is augmented to include several ReAwers. These Reducers will have to share 
Graph Memory and Process Descriptor Memofy, but to the degree that the Reducers can inter- 
leave memory cycles there will be more processes diq>ased of in any time interval. This system 
represenu a very general type (rf multiproceasOT where Aned memmy is used up to the pcrint 
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where additional processors sharing the memory is no longer benificial, after which groups of 
processor/memory units are interconnected with a communications network. 

4.5. Load Balancii^ 

It was pointed out in Section 3 that because a node is always reduced by the PE in whose 
memory it resides, a policy for allocating new nodes to PBs is equivalent to a policy for distribut- 
ing the workload. The distribution of workload is mainly an issue in the topological layer, for it is 
only the communications network that can 'see* all the PEs and thereby have an indication of 
which PEs are lightly loaded and which are heavily loaded. 

Load balancing is accomodated by changing the OKEATB-REQ message so that is not 
directed at any particular PE. The c<mimunications network. ^>on obtaining a CREAIVJIEQ 
message, can route it to the PE that is the least loaded. Since the CREATB-ACK message contains 
a complete pointer, including PE number, no npceial support is required from the issuer of the 
CREATE-REQ message. 

In general, two different types of CKEATE4tEQ menages will have to be provided: <me fm 
nocfes that are to be allocated on a PE to be determined by the IcmmI balancer, and one for nodes 
where the PE is specified by the PE sending the request. An instance where the latter u required 
is when a PE must allocate a node in its own memory to copy a node received in a REDUCB-ACK 
message. 

5. Comparison With Ezbtiai Proposals 

In the introduction it was stated that the parallel-semantic layer as described here is essen- 
tially the same as the parallel-semantic layers of othor parallel graph reduction machines that 
have been proposed, except that here it presented more syttenuttcally and thoroughly. The other 
proposals will now be compared to the system here. 
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5 J. Keller, LiBditreai, aad PatU 

Perhaps the most detailed deicrqption of a parallel grq>h reduction machine is given by 
Keller et. a/.^ and while their machine differs frcnn the scheme here in minor ways, it fits the 
abstract architecture quite weU. 

The FOL language that their machine uses reflectt their machineli load balancing policy: all 
nodes belonging to a single user procedure are allocated <m the same PE. A code block in their 
system is a type of constant, and the Invoke qperattv executes by using the information in a code 
block to create a ecXLtcXiaa oi nodes (all on one PE). Some of the nodes created by the Invoke will 
include information computed at run time in additicm to the compile time information taken from 
the code block. This and many other issues discuned in the Keller paper actually pertain to the 
sequential-semantic layer rather than the parallel-cemantic layer. 

Other aspecU of their machine are quite familiar. Their machineli 'demand-list' and 'result- 
list* are similar to the process descriptors at the abstract ma^ne. In Keller^ machine, however, 
notifiers are associated with each node, rather than with each ^ocess (tadc, in their terminology), 
and are preassigned in most one*. This is possibte because they (mly attempt to exploit subgraph 
sharing within a user function definition, and so most aottters are available at compile time. 
There is really no advantage in precomputiog tte aMifiers, and leaving qtace in each node fen: a 
notifier is wasteful of q>ace doce only a fraction of the nodes at any time will be in the 'reducing' 
state. Inclutfing the ncrtifiers in the nodes also forea thdr qrstem U> use 'forward chaining' to 
handle multiple global notifiers. While thto tedmiqoe has dw advantage that the tpuce for 
notifiers is not ci variabte size, it increases tlte amount of coomraaieatimi necessary, for in addi- 
tion to the actual notification messages, their system requtm adtitkmal messages to set up the for- 
ward chaining. No real menuwy qwce is saved, for the same nvmber of notifiers must be »tarcd in 
either system. 

Keller^ paper gives no detailed discusuon of what messages are passed in his system, so no 
comparison of communicatitm semantics » posdble. 
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52. DarUngtm amd Rccvt 

The ALI(^ multi-proccMoc' U vety interesting because at first glance it appears to be 
greatly different from the machine described here. As in KeUer^ machine, nodes of the graph 
contain notifiers in addition to the information contained in nodes ci the abstract machine. In 
ALICE, however, the nodes are all put in a shared memtvy to which each of the PEs has access. 
Darlington recognizes that shared memory limits the number of PEs that can successfully be 
employed in this way, so he proposes connecting groups ot memory/PE units with a communica- 
tion network. 

This, of course, is the scheme discussed in Section 44, wherein multiple Reducers are pro- 
vided in each PE. In Section 44, the Reducers had to riiare common resources, including the 
memory itself, the Computation Message Processor, and the Storage Message Processor. These 
common services are also described in Darlingtcm^ papn; there, he vimalizes the stream ai runn- 
able processes and the free node list as 'constantly circulating dotted ccmmunications rings'. 

Darlington also points out that when PE groups are coinected by a communication network, 
the network serves to 'map the local memtvies cmto the ^obal address qiace of the system". This, 
of course, is reflected in the (PE address) form that pmnters take in the system here. Darlington 
goes on to say that the communication netwcvk k used to shure fwocessable nodes and free space 
among the building blocks. While the latter is certainly trae - this is the load balancing function 
described in Section 45 -- the fonact contra<ficts his eulier statement, f<Mr the mapping of local 
memfiries into the global address space precludes the migration of nodi» frmn (me memcwy unit to 
another. Such migration is possible if forwarding addresses ue left behind or if the cmnmunica- 
tion netw(»k serves to translate "virtual addtetaei/' q>pearing in nodes to 'physical addresses' con- 
sisting oi PE/adikess pairs, but the f cmner entaik commioicatkMi overhead to perform the f ot- 
warding, and the latter turns the communicaticm netwoit into a huge bottleneck through which all 
memory references must pass. In particular, any benefit that might be (Atained from grouping 
related nodes into the same memmy segment is lost. 
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Abandoning the extremely inefficient feature ol allowing tlie migration of unreduced or par- 
tially reduced nodes, then, bringi ALICE on par with the abstract architecture presented here. 
The main difference is that in Darlington^ paper, a diared memory system is the starting point 
from which a hybrid shared memory/message passing system is developed. Here, a message past- 
ing model is the starting point frcmi which the hybrid is easily derived (in Section AA). 
Darlington'k paper provides no ^taib of what communicati<m takes place in the hybrid version of 
ALICE. 

The last major difference between the ALICE machine and the abstract machine presented 
here is that ALICE supports the accessing of nodes, tot both reading and writing, that have not 
been reduced. This is in opposition to the principles set forth in Section 2, and reflects the fact 
that ALICE is capable of supporting base languages oihet than strictly constant applicative form 
languages. Whether this fact presents any special problems is a topic tor future research. 

53. Sleep and Bnrtm 

Sleep and Burton give a very brief description <rf a parallel reduction machine' that uses a 
form of combinator code as a base language. Most of their paper deals with the propertia of 
base languages and with the details of their communication network, and so there is little to com- 
pare with the system here. What little they do (fiscuss of the parallel-semantic layer is quite fami- 
liar; in particular, they describe the use of the status field of nodes. 

i. CuBcluioas 

Many parallel graph re<foction machines have been prc^Mised, but little has been done to 
establish the operating principles common to aU such machines. The work here attempts to sys- 
temize the derign of parallel reduction machines by dividinf the topic into three layers: the 
sequential-semantic layer, the parallel-semantic layer, and the top<d<^al layer. The parallel- 
semantic layer, it turns out, embodies the fundamental essence of parallel reduction in the 
abstract; as such, the parallel-semantic layers of dl parallel redncticm machines will be similar, if 
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not identical. 

The parallel-femattc layer hai been described here to a sufficient level of detail that only the 
language and communicatitm network would need to be designed to create a complete machine. 
In particular, the aspects covered in the parallel-semantic lay» include the overall structure of the 
machine, the semantics ot the messages that travel the ccnnmunications network, the data struc- 
tures maintained by the processing element, and the alpnithms necessary to manage these data 
structures. The correctness of the scheme presented here was demonstrated by an emulation pro- 
gram written for a Symbolics 3600 Lisp Machine. 

While other groups have proposed parallel reductioa machines, no proposal has described the 
parallel-semantic layer of a machine to the degree ot detaU as with the abstract machine 
presented here. To the degree that these other machines are <fescribed. their parallel-semantic 
layers are consistent with the model here. But the architecture presented here is more than a 
hypothetical machine; by providing an abstract model tot parallel graph reduction, it u hoped that 
insight into the parallel reduction process itself can be gained. Such insight will undoubtedly 
prove useful in the design and construction of actual high-perfcMmance graph reduction machines. 



37 



REFERENCES 



(1) T J.W. Qarke, ?JS. Gladstone, CD. Macleui. and AC Nrnman, "SKIM - The S, K, I Reduc- 
tion Machine', Froe. 1980 LISP Conference, Augntt 1980. 

(2) J. Darlington and M. Reeve, 'ALICE: A Multi-Proceaaor Rediictt<m Machine for the Parallel 
Evaluation ctf Applicative Languagea', Proeeedti^s tf tke 1981 Cenieremce oh FwictioHol Pro- 
gramming Languages and Cmi^mer ArcUtectnre, 1981, pp. 65-76. 

(3) T. Johnsson, The O-Machine: An AlMtract Machine Cor Graph Reduction', Programming 
Methodology Group, Department of Computer Science, Chalmert Univenity of Techonology, 
S-412 96 Goteborg, ^eden. 

(4) R. M. Keller, G. Lin<ktrom, and S. Patil, 'An Architecture for a Loosely-coupled ParaUel Pro- 
cessor". Tech. Report UUCS-78-lItf, Univerdty of Utah, October 1978. 

(5) G. A. Mago, 'A Cellular Computer Architecture for Functional Programming'. COMPCON 
Spring 80, February, 1980, pp. 179-187. 

(6) M. R. Sleep and F. W. Burton, "Towarcfa a SEero Asngnawnt Parallel Processor*, Proceedings 
of the Second Inienuaional Cmtferenee en Distribnied Con^^mting Systems, 1980, pp. 80-84. 

(7) D. A. Turner, 'A New Implementati<m Technique for Aj^licative Language**, Software - 
Practice and Experience, 9(1979). pp. 3M9. 



